Video overlay device of mobile telecommunication terminal

ABSTRACT

A video overlay device of a mobile telecommunication terminal comprises a multiplexer for outputting at least one of a first video data and a second video data; and a video overlay unit for overlaying at least one of the first and second video data with graphic data in accordance with a predetermined ratio. The multiplexer selectively outputs at least one of the first and second video data based on input video selection signals provided by a central processing unit (CPU).

CROSS-REFERENCE TO RELATED APPLICATION

Pursuant to 35 U.S.C. § 119(a), this application claims the benefit ofearlier filing date and right of priority to Korean Patent ApplicationNo. 2002-78469, filed on Dec. 10, 2002, the content of which is herebyincorporated by reference herein in its entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a mobile telecommunication terminal,and more particularly, to a mobile telecommunication terminal which candisplay graphic data displayed on a liquid crystal display (LCD) screenof a mobile telecommunication terminal.

2. Description of the Related Art

A mobile telecommunication terminal can be utilized to provide variousfunctions such as a camera function and a replaying function in additionto voice communication functions. The camera function and the multimediareplaying function are more popular how that wideband services isavailable. A mobile telecommunication terminal may include a highresolution digital cameral and a color LCD screen.

On the LCD screen of the mobile telecommunication terminal, iconsprovide certain information such as receiving sensitivity, batterycapacity, message, bell/vibration, etc. Certain graphic data for a menudisplay and video input through a camera and streaming video signalsdecoded by an MPEG-4 codec (coder and decoder) are also displayed.

Referring to FIG. 1, a block diagram showing a construction of a generalmobile telecommunication terminal in which various video data can bedisplayed is provided. A general mobile telecommunication terminalcomprises a central processing unit (CPU) 10; a camera module 102; anMPEG-4 video codec 103, and an LCD 104. The CPU 10 is for controllingeach of the above components. The camera module 102 is for photographingvideos as a digital method by using an image sensor. the MPEG-4 videocodec 103 is for compressing and restoring graphic data inputted fromthe CPU 101, MPEG-4 streaming video data, and video data inputted fromthe camera module. The LCD 104 for displaying the compressed andrestored data

The MPEG-4 video codec 103 includes a host interface 1031, a camerainterface unit 1034, a video processing unit 1032, an arbitrationcircuit unit 1033, a memory 1035, and an LCD interface unit 1036. Thehost interface 1031 is for giving and taking graphic data and controlsignals with the CPU 101. The camera interface unit 1034 is forreceiving a digital video signal from the camera module 102.

The video processing unit 1032 is for compressing the digital videosignal inputted from the camera module 102 by a standard of the MPEG-4or restoring MPEG-4 streaming data inputted from the CPU 101. Thearbitration circuit unit 1033 is for arbitrating operations of saidrespective devices and data processing processes. The memory 1035 is forstoring video data inputted from the camera module or video datarestored in the video processing unit; and an LCD interface unit 1036for providing data and control signals to the LCD 104.

Also included are a first display layer for displaying each kind of menuand icon showing various information of the terminal, a second displaylayer for video data photographed through a camera module, and a thirddisplay layer for video data which have restored MPEG-4 video streamingdata downloaded from a video server.

The first layer provides for display for menus, icons, and a backgroundscreen of the mobile telecommunication terminal. The graphic data isdisplayed on the LCD through the host interface 1031, the arbitrationcircuit unit 1033, and the LCD interface 1036 on the basis of datashowing information of the mobile telecommunication terminal providedfrom the CPU 101 and a clock and a synchronization signal required tooperate the mobile telecommunication terminal.

The second layer provides for display of video data inputted from thecamera module and is to store video data inputted from the camera moduleinto the memory 1035 by compressing as one image, or to display inputtedvideo data on the LCD 104 in real time. Real time digital R, G, B (red,green, and blue) signals inputted from the camera module 102 aredisplayed on the LCD 104 through the camera interface 1034, the videoprocessing unit 1032, the arbitration circuit unit 1033, and the LCDinterface 1036. Also, images stored in the memory 1035 are inputted tothe CPU 101 through the arbitration circuit unit 1033 and the hostinterface 1031, compressed into a certain form (for example, JPG), andthen stored in the memory 1035.

The third layer provides for the display of video data which haverestored the MPEG-4 streaming video data is to display MPEG-4 streamingvideo data inputted from a video server (not shown) by restoring asoriginal video data. The inputted video data is inputted to the videoprocessing unit 1032 from the CPU 101 through the host interface 1031and the arbitration circuit unit 1033. At this time, the videoprocessing unit 1032 outputs the inputted MPEG-4 streaming video data byrestoring as original video data. Also, the arbitration circuit unit1033 receives the restored data and displays on the LCD 104 through theLCD interface 1036.

The three display layers have the following problems. When graphic datasuch as icons and the menus and video data are to be displayed, icons(play, stop, mute, and etc.) for controlling the video data andinformation of the terminal (for example, receiving sensitivity, batterycapacity, and etc.) are displayed on an upper side or a lower side ofthe LCD screen. This limits the display area for video display.Alternatively, if the data is displayed on the entire LCD screen of themobile telecommunication terminal in order to solve said problem, thenthe icons and menu information of the terminal corresponding to graphicdata are not displayed. A solution is needed to the above problems andto maximize the video display area.

SUMMARY OF THE INVENTION

In accordance with one embodiment, a video overlay device of a mobiletelecommunication terminal comprises a multiplexer for outputting atleast one of a first video data and a second video data; and a videooverlay unit for overlaying at least one of the first and second videodata with graphic data in accordance with a predetermined ratio. Themultiplexer selectively outputs at least one of the first and secondvideo data based on input video selection signals provided by a centralprocessing unit (CPU).

The video overlay unit comprises a color space convert (CSC) unit forconverting at least one of the first and second video data into colorsignals; a color look-up data structure comprising information forconverting graphic data representing device information into colorsignals; a color palette unit for blending video data converted in theCSC unit with graphic data converted based on the information in thecolor look-up data structure; and a video selection unit for selectivelyoutputting at least one of the first and second video data converted inthe CSC unit, blended video and graphic data, and the graphic dataconverted based on the color look-up data structure.

The color palette unit blends at least one of the first and second videodata with the graphic data according to the predetermined ratio. Thepredetermined ratio is determined based on the video blend signals,provided by the CPU. The video selection unit selectively outputs atleast one of the first video data, the second video data, and theblended video and graphic data.

The first video data is provided by a camera module mounted to themobile telecommunication terminal. The second video data comprises MPEG4streaming video data downloaded from a video server. The MPEG-4streaming video data is restored using an MPEG-4 codec.

In accordance with another embodiment, a video overlay device of amobile telecommunication terminal comprises a central processing unit(CPU); a camera module for capturing videos; a video codec forcompressing and restoring streaming video data provided by the cameramodule; a multiplexer for selectively outputting the streaming videodata provided by at least one of the camera module and the video codec;and a video overlay unit for overlaying video data provided by themultiplexer with graphic data provided by the CPU, according to apredetermined ratio to produce overlaid data.

Some embodiments comprise an LCD interface unit for providing theoverlaid data to a display device or an LCD for displaying the overlaiddata provided by the LCD interface unit. The multiplexer selectivelyoutputs the streaming video based on video selection signals provided bythe CPU. The video overlay unit comprises a color space converter (CSC)unit for converting the video data into color signals.

In some embodiments, the video overlay unit further comprises a colorlook-up data structure for converting graphic data associated withterminal related information into color signals. The video overlay unitcan further comprise a color palette unit for blending video dataconverted in the CSC unit with graphic data converted according to thepredetermined ratio.

In some embodiments, a video selection unit is included for selectivelyoutputting at least one of the video data converted in the CSC unit,blended video and graphic data provided by the color palette unit, andthe graphic data converted based on information included in a colorlook-up data structure in is the video overlay unit. The color paletteunit blends at least one of the first and second video data with thegraphic data according to the predetermined ratio provided by the CPU.

The predetermined ratio is calculated based on the video blend signalsprovided by the CPU. The video selection unit selectively outputs atleast one of video data provided by the camera module, video datarestored through the codec, the graphic data, and blended video andgraphic data based on selection signals provided by the CPU.

The camera module digitally captures videos by way of an image sensor.The video codec is an MPEG-4 video codec for compressing and restoringMPEG-4 streaming video data and video data provided by the cameramodule.

The present invention is directed to a video overlay device of a mobiletelecommunication terminal which can overlay graphic data and video databy blending graphic data displayed on an LCD screen of a terminal withvideos inputted through a camera or video data restored by an MPEG-4codec with a desired ratio.

These and other embodiments of the present invention will also becomereadily apparent to those skilled in the art from the following detaileddescription of the embodiments having reference to the attached figures,the invention not being limited to any particular embodiments disclosed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this specification, illustrate embodiments of the invention andtogether with the description serve to explain the principles of theinvention.

FIG. 1 is a block diagram of a mobile telecommunication terminal inwhich a graphic display and a video display are possible in accordancewith the related art;

FIG. 2 is a block diagram of a mobile telecommunication terminalequipped with a video overlay device according to one embodiment of theinvention;

FIG. 3 is a block diagram of a minute construction of a video overlayunit according to one embodiment of the invention;

FIG. 4 is an exemplary view for explaining operation characteristics ofeach control signal according to one embodiment of the invention;

FIG. 5 is an exemplary view for explaining an output method of videoaccording to one embodiment of the invention; and

FIG. 6 is a video blend table showing a video blend ratio according toone embodiment of the invention.

Features, elements, and aspects of the invention that are referenced bythe same numerals in different figures represent the same, equivalent,or similar features, elements, or aspects in accordance with one or moreembodiments of the system.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to FIG. 2, a mobile telecommunication terminal in accordancewith one or more embodiments of the invention comprises a centralprocessing unit (CPU) 201; a camera module 203; an MPEG-4 video codec204; a multiplexer 205; a video overlay unit 206; a memory unit 207; anLCD interface unit 208; and an LCD 209. The central processing unit(CPU) 201 controls each component; the camera module 203 captures videosby using an image sensor; the MPEG-4 video codec 204 compresses andrestores MPEG-4 (motion picture experts group) streaming video data andvideo data inputted from the camera module.

In some embodiments, the multiplexer 205 outputs video signals from thecameral module 203 and the MPEG-4 video codec 204. The video overlayunit 206 provides video data from the multiplexer 205 and graphic datafrom the CPU 201 with a predetermined overlay ratio. The memory unit 207stores the overlaid data. The LCD interface unit 208 is for outputtingthe overlaid data to a display device. The LCD 209 is for displayingdata outputted from the LCD interface unit 208.

In accordance with one embodiment, the CPU 201 controls each systemcomponent above, including the multiplexer 205, outputs graphic datashowing information of the mobile telecommunication terminal, andrestores the MPEG-4 streaming video data through the MPEG-4 codec. Thisprovides the output to the multiplexer 205. Also, videos inputted fromthe camera module 203 are processed by the MPEG-4 codec 204 and therebyare outputted to the multiplexer 205.

The multiplexer 205 receives video data from the camera module 203 andthe MPEG04 codec 204, and outputs one video data on the basis of controlsignals outputted from the CPU 201. The video overlay unit 206 blendsvideo data outputted from the multiplexer 205 and graphic data outputtedfrom the CPU 201 with the predetermined ratio on the basis of controlsignals outputted from the CPU 201.

The predetermined ratio is calculated based on a basis blend table.Also, the blending video data is stored in the memory 207 or outputtedto the LCD interface 208, in certain embodiments. Accordingly, the LCDinterface 208 receives blending video data from the video overlay unit206 and thereby displays the data on the LCD 209 of the mobiletelecommunication terminal. Also, the LCD interface 208 outputs controlsignals for controlling the LCD 209 of the mobile telecommunicationterminal.

Referring to FIG. 3, the video overlay unit comprises a CSC (color spaceconvert) unit 301; a look-up table 302; a color palette unit 303; avideo selection unit 304, and an LCD interface 305. The CSC (color spaceconvert) unit 301 converts camera video data outputted through themultiplexer 205 and restores MPEG-4 video data (16 bits) into color data(R, G, B 24 bits).

In some embodiments, the look-up table 302 is provided for receivinggraphic data from the CPU 201 and thereby converting into color data (R,G, B 24 bits). Other data structures can be implemented to provide thesame service. The color palette unit 303 is for blending video dataoutputted from the CSC unit 301 with graphic data outputted from thecolor look-up table 302 with a predetermined ratio. The video selectionunit 304 is for selectively outputting one of the video data of the CSCunit 301, the video data of the color palette unit 303, and the graphicdata of the color look-up table 302. The LCD interface 305 is foroutputting data outputted from the video selection unit 304 and eachkind of synchronization signal required for an LCD display.

Referring to FIG. 4, the control signals, i.e., input video selectionsignals (VS_SEL), video blend signals (B_L), and output video selectionsignals (VID_SEL) are respectively outputted through a general purposeinput output (GPIO) port of the CPU 201 in order to control themultiplexer 205, the color palette 303, and the video selection unit304.

In one embodiment, the input video selection signals (VS_SEL) areinputted to the multiplexer 402. The multiplex selects one of video dataoutputted from the camera module and video data restored through theMPEG-4 thus to output. The video blend signals (B_L) are inputted to thecolor palette 4031 in order to control a blend ratio of the video dataand the graphic data.

The output video selection signals (VID_SEL) are inputted to the videoselection unit 4032. The video selection unit 4032 selectively outputsone data of the restored MPEG-4 video data, the camera video data, andthe graphic data, or video data where the video data and the graphicdata are blended.

Referring to FIG. 5, a method for displaying the restored MPEG-4 videodata or the camera video data is provided. The multiplexer 205 receivesdigital video data outputted from the camera module 203 and the MPEG-4codec 204. The multiplexer 205 selects one of video data inputted fromthe camera module 203 and video data inputted from the MPEG-4 codec 204.

The selection is based on the input video selection signals outputtedfrom the CPU 201 to provide an output to the video overlay unit 206. Thedigital video data outputted from the camera module 203 and the MPEG-4codec 204 is YUV (brightness, color) data (16 bits), and inputted to theCSC unit 301 thus to be converted into R, G, B data (24 bits), forexample.

In certain embodiments, the video selection unit 304 selects one videodata based on the output video selection signals (VID_SEL) of 2 bitsinputted from the CPU 201, and outputs it to the LCD interface 208.Also, the LCD interface 208 outputs control signals of a video pixelclock (VCLK), a vertical synchronous signal (Vsync), and a horizontalsynchronous signal (Hsync) and 8-bit video data (one 24-bit pixel dataper 3 cycles) to the LCD 209 and thereby displays.

The color look-up table converts graphic data (16 bits) inputted fromthe CPU 201 into R, G, B data (24 bits). The video selection unit 304selects the graphic data on the basis of the output video selectionsignals (VID_SEL) of 2 bits (0,0) inputted from the CPU 201, and outputsit to the LCD interface 208. Also, the LCD interface 208 outputs controlsignals of a video pixel clock (VCLK), a vertical synchronous signal(Vsync), and a horizontal synchronous signal (Hsync) and 8-bit videodata (one 24-bit pixel data per 3 cycles) to the LCD 209 and therebydisplays.

In one or more embodiments, the color palette unit 303 receives therestored MPEG-4 data outputted from the multiplexer 205 or data whichthe camera video data is converted into R, G, B data through the CSCunit 301, and receives data which graphic data outputted from the CPU201 is converted into R, G, B data through the color look-up table 302.Advantageously, the color palette unit blends data inputted from the CSCwith data inputted from the color look-up table with a predeterminedratio on the basis of the output video selection signals (VID_SEL) of 2bits (1,0).

Referring to FIG. 6, the color palette unit 303 determines a blend ratioof the inputted data with a predetermined ratio of a predetermined videoblend table on the basis of 4-bit video blend signals (B_L) outputtedfrom the CPU 201. Accordingly, the color palette 303 blends video datainputted from the CSC 301 with graphic data inputted from the colorlook-up table 302 with the predetermined ratio based on the video blendtable, and outputs to the video selection unit 304.

Preferably, when a bit combination of the video blend signal (B_L) is‘0,1,0,0’, for example, a blend ratio of the graphic data is 100% and ablend ratio of the video data is 0%. As such, the LCD of the mobiletelecommunication terminal displays the graphic data. When a bitcombination of the video blend signal (B_L) is ‘0,1,0,1’, for example, ablend ratio of the graphic data is 0% and a blend ratio of the videodata is 100%. Accordingly, the LCD of the mobile telecommunicationterminal displays only the video data.

In one embodiment, when a bit combination of the video blend signal(B_L) is ‘1,0,1,0’, for example, a blend ratio of the graphic data is25% and a blend ratio of the video data is 75%. Accordingly, the LCD ofthe mobile telecommunication terminal displays the graphic data and thevideo data in a state that the video data is a little overlaid by thegraphic data.

The video selection unit 304 selects blending data of the graphic dataand the video data on the basis of the output video selection signals(VID_SEL) of 2 bits (1,0) inputted from the CPU 201, and thereby outputsto the LCD interface 208. Also, the LCD interface 208 outputs controlsignals of a video pixel clock (VCLK), a vertical synchronous signal(Vsync), and a horizontal synchronous signal (Hsync) and 8-bit videodata (one 24-bit pixel data per 3 cycles) to the LCD 209.

In accordance with one embodiment of the invention, the video overlaydevice of a mobile telecommunication terminal according to the presentinvention can overlay graphic data and video data by blending graphicdata displayed on an LCD screen of the terminal with videos inputtedthrough a camera or MPEG moving pictures with a desired ratio. The videooverlay device of a mobile telecommunication terminal displays movingpictures on the entire LCD screen of the terminal and at the same timeoutputs each kind of control icon or icon for displaying a state of theterminal by overlaying. According to this, the user can appreciate themoving pictures in a wide screen and know information of the terminal,thereby having enhanced convenience.

The embodiments described above are to be considered in all aspects asillustrative only and not restrictive in any manner. Thus, otherexemplary embodiments, system architectures, platforms, andimplementations that can support various aspects of the invention may beutilized without departing from the essential characteristics describedherein. These and various other adaptations and combinations of featuresof the embodiments disclosed are within the scope of the invention. Theinvention is defined by the claims and their full scope of equivalents.

1-23. (canceled)
 24. A mobile communication terminal comprising: acamera adapted to capture an image; a video codec for compressing andrestoring data and configured to communicate with at least one of thecamera and a memory; a processor adapted to communicate with at leastone of the memory and the camera to selectively output first type datain response to information provided by at least one of the camera andthe memory; and a video overlay unit for at least partially combiningthe first type data and a second type data provided by the processoraccording to a predetermined ratio to produce a third type data, whereinthe video overlay unit converts at least part of the first type data tocolor signals and blends with at least part of the second type data. 25.The mobile communication terminal of claim 24, wherein the video overlayunit comprises: a color space converter (CSC) unit for converting thefirst type data into color signals, and a color palette unit forblending the first type data converted in the CSC unit with the secondtype data converted according to the predetermined ratio.
 26. The mobilecommunication terminal of claim 24, wherein the first type datacomprises one of still image data and video data.
 27. The mobilecommunication terminal of claim 24, wherein the second type datacomprises graphic data.
 28. The mobile communication terminal of claim24, wherein the third type data comprises overlaid data that at leastpartially overlays the first type data with the second type data. 29.The mobile communication terminal of claim 24, wherein the video codecand video overlay device are integrally provided in the processor. 30.The mobile communication terminal of claim 24, wherein the video overlayunit blends at least part of the first type data with at least part ofthe second type data according to the predetermined ratio.
 31. Themobile communication terminal of claim 24, wherein the predeterminedratio is determined based on a video blend signal.
 32. The mobilecommunication terminal of claim 25, wherein the video overlay unitfurther comprises: a video selection unit for selectively outputting atleast one of the first type data converted in the CSC unit, blended dataprovided by the color palette unit, and the second type data convertedbased on information included in a color look-up data structure in thevideo overlay unit.
 33. The mobile communication terminal of claim 32,wherein the video selection unit selectively outputs at least one ofimage data provided by the camera, data restored through the codec, thesecond type data and the third type data based on selection signals. 34.The mobile communication terminal of 24, wherein the video codeccomprises an MPEG-4 video codec.
 35. A method of displaying image on amobile communication terminal with a camera, the method comprising:configuring a video codec, which is for compressing and restoring data,to communicate with at least one of the camera and a memory; selectivelyoutputting first type data from at least one of the camera and thememory; and at least partially combining the first type data and asecond type data provided by a processor according to a predeterminedratio to produce a third type data, wherein at least part of the firsttype data is converted to color signals and blended with at least partof the second type data.
 36. The method of claim 35, wherein the firsttype data comprises one of still image data and video data.
 37. Themethod of claim 35, wherein the second type data comprises graphic data.38. The method of claim 35, wherein the third type data comprisesoverlaid data that at least partially overlays the first type data withthe second type data.